Last hour you built a filter. This hour you will use it — on your own problem, with a classmate holding the pen.
Diagnose and repair a research problem before it costs you a semester.
Short bursts of theory + error clinic. You will be scored by a peer, twice.
Your Research Notebook, open at Section 3 — Problem Statement.
One repaired sentence, one killed idea, one dated notebook entry.
Dr. Mohsin Furkh Dar · School of Computer Science · 5th Semester
B.Tech CSE · 55 minutes
Navigate: → / Space next · ← back · O slide list ·
D dark mode · F full screen
A research problem is a gap you could be wrong about.
Gaps come from five findable places. Search, not spark.
Narrow with the funnel and the five knobs until ≈1 project fits.
FINER + M decides whether the gap is worth two semesters.
Most failures are five predictable errors — and each has a known repair.
What do the six letters stand for, and what is the one rule about how they combine?
Lecture I.4 gave you a filter. A filter tells you that something is wrong. It does not tell you what is wrong or what to do about it.
Today is a catalogue of failures and a repair manual. It is diagnostic, not conceptual.
Zillow is the largest property listing site in the US. Its "Zestimate" price model had been refined for a decade; on homes actually on the market, its reported median error was in the low single digits of percent.
In 2018 the company asked the obvious next question: if we can price a house better than anyone, why not buy houses ourselves, hold them briefly, and resell? Algorithmic house-flipping at national scale.
The research problem they solved: “Can we predict a home’s market value from its features and comparable sales with low median error?”
Answer: yes. Demonstrably, repeatedly, at scale.
Thirty seconds with the person next to you. The model was accurate and the company still shut the business down in November 2021.
What went wrong?
Rule: "the model was overfitted" is not an acceptable answer in this course. Tell me what the problem statement failed to contain.
reported inventory write-down in one quarter (Q3 2021) — homes worth less than Zillow paid for them.
of the workforce cut as the iBuying arm was wound down, announced November 2021.
experiments that could have told them earlier — because no result had been pre-declared as failure.
| The problem that was solved | The problem that mattered |
|---|---|
| “Predict a home’s market value with low median error.” | “Can we buy at prices that stay profitable after holding, renovation and resale — when our own offers move the local market and the cost of an error is asymmetric?” |
| Error measured symmetrically: over-estimating by 5% and under-estimating by 5% score identically. | Over-paying costs real cash on thousands of houses. Under-paying costs only a declined offer. The metric never knew that. |
| Validated on homes listed for sale. | Deployed on homes Zillow itself bought — a different population, selected partly by the model's own errors. |
The model answered its question correctly. The question was the defect.
You will not lose $304 million. You will lose a semester — and you will lose it to one of five errors that are visible in the first sentence you write.
Solved: can a CNN grade diabetic retinopathy as accurately as an ophthalmologist? Yes, ~90% in the lab.
Missing: clinic lighting, intermittent internet, a nurse's queue. Over 1 in 5 images rejected; patients sent home.
Constraints not inside the statement
Framed as: “use AI to help cure cancer.” Which cancers? Which decision? Against which standard?
Missing: anything falsifiable. Reported spending on the order of $62M at one site before the project was shelved in 2017.
No scoreboard, so nothing was ever settled
Solved: price prediction, accurately.
Missing: the construct that mattered (profit under asymmetric cost and feedback) was never the thing being measured.
The metric was won without the construct
In none of the three was the failure a failure of engineering skill. In all three, the problem was selected and stated badly — and that was visible on day one, in one sentence, for free.
| 0–8 | Zillow autopsy + the waiting-room vote |
| 8–16 | Attributes of a well-posed problem; three free tests |
| 16–32 | The seven errors, each with its repair |
| 32–38 | Diagnosis drill (scored) + repair matcher |
| 38–50 | Error clinic — peer diagnosis of your own statement |
| 50–55 | Debrief, notebook task, next lecture |
Roughly half this hour, you are working — not listening.
Six real-shaped statements from final-year project proposals. Click the one you would most willingly supervise, then click the one you would refuse first. Argue with your neighbour before revealing.
All six are unsupervisable — and each fails in a different, named way.
That is the design of this lecture. Statement 1 is your over-scoping example, 2 is unfalsifiable, 3 is already solved, 4 is resource-blind (and ethically stopped), 5 is method-first, 6 is implausible. We now take them one at a time, and we repair each one on this screen.
Click each attribute to see the question a reviewer actually asks. Errors are simply attributes that are missing.
Pick an attribute above.
Hand the sentence — only the sentence — to a classmate. Ask them to write down: dataset, baseline, metric, threshold, population.
Every blank is a defect. You have the missing information in your head; a reviewer only ever gets the sentence.
Complete this sentence out loud: “I would conclude I was wrong if …”
If you cannot finish it in ten seconds, the problem is unfalsifiable — and no amount of work will fix that later.
Could you produce a crude end-to-end result in one week? Bad numbers, tiny sample, ugly code, one seed — but end to end.
If no, the problem is probably not feasible at your scale. Coding is usually under a third of the total effort.
A “1” found in week 3 costs you an afternoon.
The same “1” found in week 12 costs you the semester.
This is the entire argument for the error clinic at the end of this hour.
| # | Error | Symptom in the sentence | Criterion it breaks | The only repair that works |
|---|---|---|---|---|
| E1 | Over-scoping | No dataset, no baseline; tens of thousands of search hits | F | Narrow — population knob first |
| E2 | Unfalsifiable claim | “improve”, “enhance”, “optimise” — no construct, no threshold | M, then R | Re-measure — construct → definition → metric → threshold |
| E3 | Already-solved problem | The answer is in a textbook; “no recent papers” was read as “open” | N, R | Re-anchor — a context nobody has measured |
| E4 | Resource-blind choice | Needs data, compute or approvals you cannot get in time | F | Re-anchor to obtainable data — narrowing never fixes access |
| E5 | Method-first framing | The sentence starts with a technique; nobody changes anything if it works | R, then I | Re-anchor to a person's pain; let the method follow |
| E6 | Implausible result | “99% accuracy on stock prices” — if it worked you would not be publishing it | F, N | Re-measure — turn it into a validity or replication question |
| E7 | Metric shopping | The dataset or measure changes until the result looks good | E (integrity) | Abandon the practice — pre-commit the metric before running |
Only four repair moves exist. The move that is not on the list is the popular one — keep the problem and hope the criterion stops mattering. It reappears at the review panel instead.
Why students resist the fix: narrowing feels like admitting the project is small. It is the opposite — a narrow question is the only kind that can be answered, and an answered question is a contribution. A broad one is a wish.
BEFORE
“AI for healthcare in India.”
AFTER
“For diabetic-retinopathy screening, does a publicly available pretrained classifier reject more than 15% of smartphone-fundus images captured under uncontrolled clinic lighting as ungradable, and does a low-light enhancement step reduce rejections without dropping referable-DR sensitivity below 0.85?”
Longer? Yes. Answerable in one semester? Also yes — and every noun in it tells you what to do on Monday morning.
Turn the population knob first, because nobody else will have turned it the way you do. That is where student novelty legitimately lives.
Each dropdown is one of the five knobs from Lecture I.3. Turn too few and you have a topic; turn all five and you have engineered away everything interesting.
YOUR STATEMENT
Does [task] differ [from what], measured by [metric], [for whom], [under what conditions]?
Knobs turned: 0 of 5
More than 100 distinct projects would satisfy this sentence. That is a field, not a question.
Calibration rule (Lecture I.3). Ask: how many distinct final-year projects would satisfy this sentence?
>100 = too broad · 5–20 = a topic · ≈1 = you have a question · 0 = over-narrowed, nobody cares.
improve enhance optimise user-friendly efficient robust smart modern technologies better performance
Each names a construct with no operational definition. Nothing could count as failure — therefore nothing counts as success either. You can work for four months and be unable to say whether you succeeded.
Two gaps to watch (Lecture I.4). The validity gap — does the metric still mean the construct? And the gaming gap — can the metric be won without the construct? Zillow lost to the second one.
BEFORE
“We will improve the user experience of the college ERP using modern web technologies.”
AFTER
“Across three core ERP tasks — register for a course, download a grade card, apply for leave — does a redesigned navigation raise unaided task-completion rate by at least 15 percentage points and cut median time-on-task below 90 s, for n = 30 second-year students?”
| Construct | Operational definition | Metric | Threshold |
|---|---|---|---|
| “usable” | task completed unaided | completion rate, time-on-task | ≥ 85%, ≤ 90 s |
“I searched and found no recent papers on this. So it must be an open problem.”
Absence of papers usually means closed, not open.
A problem with no recent literature is in one of three states, and you must find out which before you commit:
Distinguish them by reading the surveys and the textbooks, not by counting recent papers.
BEFORE
“Comparative analysis of bubble sort, insertion sort, quicksort and merge sort on random arrays.”
AFTER
“On a Cortex-M4 microcontroller with 128 KB RAM, at what input size does insertion sort stop beating introsort in energy per element (mJ/element, measured on a shunt resistor), and does that crossover shift by more than 2× between mostly-sorted and random inputs?”
Same algorithms — a different question. Nobody has published the energy crossover on your board, and energy is what matters to an embedded engineer choosing a sort routine.
This is novelty by new context — the second most respectable kind available to a student, and by far the most reachable.
Rows on my left argue FOR. Rows on my right argue AGAINST. Ninety seconds each, then I take one sentence from each side.
The discriminator is not “has it been done?” but “is the outcome already known with evidence under the conditions I will test?”
Re-running a settled textbook result on the same inputs is E3. Re-running a 2019 claim on 2026 data, on a different population, or with a correct statistical protocol, is a research contribution — and it is the kind most often accepted from students, because it is achievable and honest.
| Week 1 | “We'll get the hospital records — my uncle knows the administrator.” |
| Week 5 | “The ethics committee meets monthly. We've applied.” |
| Week 9 | “They need an institutional MoU and a data-sharing agreement.” |
| Week 12 | “Sir, can we change the topic?” |
Five resources multiply, and each can be zero: data · compute · time · skill · access. Students estimate the coding time and forget the other four. Coding is usually under a third of the total.
The rule to memorise: Narrowing does not fix this. A smaller slice of inaccessible data is still inaccessible. Narrowing fixes scope; only re-anchoring fixes access.
BEFORE
“Detecting depression from the WhatsApp chat logs of 500 students of our university.”
Two failures at once: private messages of identifiable people without consent (E · ethics, a hard stop), and data you cannot lawfully obtain (F · access).
AFTER
“On a public, de-identified shared-task corpus released under a signed usage agreement, does a classifier's recall for at-risk posts fall by more than 10 points on code-mixed posts compared with monolingual English posts?”
The mental-health framing survives; the illegal collection does not. And the limitation is declared in the statement, not discovered at the viva.
Where the ethics failure cannot be repaired — covert collection, no consent, real harm — the only move is abandon. That is not a defeat; it is a saved semester.
You have just finished a good course on transformers / GNNs / blockchain / LLM agents. The technique is the most interesting thing you own, so the sentence starts with it.
The reviewer's question you cannot answer: “Why this method?” — because the method came first and the problem was reverse-engineered to fit it.
The second question you cannot answer: “Who changes what they do if this works?”
Relevance theatre is the standard cover story: “this will help society and contribute to the nation's growth.” A sentence that fits every project fits none.
BEFORE
“Applying transformers to our canteen billing data.”
AFTER
“Can the 12:30 queue length at the canteen be predicted 20 minutes ahead to within ±8 people from three weeks of till transactions — and does a three-parameter seasonal baseline beat an LSTM at this data size?”
Notice what the repair produced for free: a person (the manager deciding how many counters to open), a threshold (±8), a baseline, and a result that is interesting either way — “the simple model wins at this data size” is a publishable, useful finding.
The so-what chain, run to the end: fewer students miss the start of the 1 p.m. lab. That is someone acting differently.
“A deep learning model that predicts the next-day NIFTY closing price with 99% accuracy.”
The free test: if it worked, would you be sitting in a classroom telling people about it? Any claim whose success would make you extremely rich, famous, or would overturn a well-tested result, deserves a very high burden of proof — and probably contains a leak.
Where such results actually come from: shuffling time-ordered data before splitting; predicting a value that is already inside the input; measuring accuracy on a direction that is 95% one class; ignoring transaction costs.
Repair · re-measure into a validity question: “Do published intraday-prediction results survive a walk-forward split with transaction costs on the same data?” — a T5 measurement/validity study, easy to defend, and genuinely useful whichever way it comes out.
“We tried five datasets and report the one where our method wins.”
Changing the dataset, the metric, the split or the seed after seeing results, until the number looks good. It is not a scoping error — it is an integrity error, and it is the one that ends careers rather than semesters.
Family resemblance: HARKing (hypothesising after results are known), p-hacking, cherry-picked baselines, unequal tuning budgets for your method and the comparison, reporting the best seed.
The protection is procedural: pre-commit the metric, the threshold and the test split in your notebook, with a date, before you run anything. That dated entry is what separates a finding from a story told afterwards. Treated fully in Unit III.
Statement 1 of 8 · Score: 0
—
Several of these commit more than one error — say so out loud when you disagree with the scoring. The argument is the exercise; the button is just a scoreboard.
Shrink scope until F is true. The default move — and the one students resist.
Change the baseline, context, population or data source. Fixes N, I, and access.
Replace the metric, keep the question. Fixes M and often R — the cheapest repair available.
The only move for a hard ethics failure. Not a defeat — a saved semester.
Case 1 of 5 · Correct: 0
—
The one hard rule: the author may not defend the statement verbally. A reviewer only ever gets the sentence. If the defence is not in writing, it does not exist.
| Criterion | Evidence the reviewer can point to in the sentence | Score |
|---|---|---|
| Feasible | Data, compute, time, skill and access all reachable by this student | |
| Interesting | Names one specific person or group who would read it and act | |
| Novel | New answer / new question / new context / new evidence — which? | |
| Ethical | Subjects, lawful acquisition, misuse, unequal harm | |
| Relevant | So-what chain terminates in someone acting differently | |
| Measurable | Construct → definition → metric → threshold, all present |
Sum: 18 / 30 · Product: 729
—
Then write, in their notebook, one sentence in this exact form:
“This is E__; it breaks __; apply __.”
Usually on M and R. And your neighbour is usually right — you have context in your head that never made it into the sentence.
Every author in this room wanted to say “no, what I meant was…”. At a review panel, in a paper submission, in a funding application — you are not in the room.
Three minutes of writing repaired a defect that would have surfaced in week twelve. That ratio — one afternoon now, one semester later — is the whole argument for this course.
“The error I am most at risk of is …”
Everyone has a signature error. Over-scopers keep over-scoping; method-first students keep starting with the method. Knowing yours is worth more than knowing all seven.
“The idea I am killing today is …”
You should leave this room with one fewer idea than you walked in with. A killed idea in week three is a result — record it, with the reason, in Section 2 of your notebook.
“Absence of papers usually means closed, not open.”
Sections touched today: 2 · Problem Ideas and 3 · Problem Statement. Do not overwrite anything — this lecture is about showing the repair, and the repair is only visible if the broken version stays on the page.
\version{2}{2026-08-18}{after Lecture I.5} and write the repaired
statement underneath. Never delete the old one.\wasnow{old phrase}{new phrase} for at least one phrase
you changed — e.g. \wasnow{improve performance}{reduce p95 latency below 100 ms}.
3 · Problem Statement
v1 (2026-08-11): “Improve crop-disease detection using deep learning.”
\version{2}{2026-08-18}{after Lecture I.5}
v2: “For three Uttarakhand crops, does a PlantVillage-trained classifier lose more than
15 points of top-1 accuracy on field-captured images, and does illumination normalisation
recover half of the drop?”
\wasnow{improve detection}{quantify the accuracy drop}
Peer review (A. Sharma): v1 = E1, lowest row M — “no metric, no threshold,
no baseline.” Move applied: narrow.
Both versions visible. Dated. Someone else's name on it. That is a research notebook; a page with only v2 on it is a fair copy.
| Candidate idea | Most likely error | Criterion at risk | Repair move | Survives? |
|---|---|---|---|---|
| Crop-disease detection for Uttarakhand farmers | E1 over-scoped | F | Narrow: 3 crops, field-captured images | Yes |
| Predicting placement outcomes from student records | E4 + ethics | F, E | Re-anchor to consented, de-identified data | Maybe |
| GNN on hostel mess attendance | E5 method-first | R | Re-anchor to the warden's actual decision | No — killed |
Then kill one idea: strike it through and write two lines saying which error it commits and why no repair move rescues it. A killed idea in week three is a result, not a loss.
We turn your repaired sentence into an executable plan: general vs specific objectives, and the box that says what you will not do. Assignment 1 (10 marks, CO1) is released in that lecture and is built directly on the statement you repair tonight — so an unrepaired statement costs you twice.
Sources: Hulley, Cummings & Browner, Designing Clinical Research (FINER) · Lazer et al. (2014), “The Parable of Google Flu”, Science (replication as contribution) · Zillow Group Q3 2021 shareholder letter and November 2021 wind-down announcement (slides 3–4) · your own Unit I decks: I.1 Thailand, I.3 five knobs, I.4 FINER+M and the four repair moves.